Foundational ideas behind what you will have to deal with during your graduate program:
Dealing with continuity (which implies infinity) \(\rightarrow\) Calculus
Two new operators:
Derivatives (Today)
Integrals (Friday)
This is often where people experience the most difficulty
We are now operating with whole functions!
New operators have new, different rules
We will stick together!
How do we find the slope of this line?
\(m = \frac{f(x_2) - f(x_1)}{x_2-x_1} = \frac{rise}{run}\)
Let’s try it out!
Positive derivative \(\rightarrow\) Function is increasing
Negative derivative \(\rightarrow\) Function is decreasing
Zeroes \(\rightarrow\) Max or min (more soon!)
\(y = x^2\)
We need an operation to calculate the slope or rate of change for (almost) any kind of continuous function
Enter derivatives
Generalize slope formula
\[ m = \frac{f(x_1+ \Delta x) - f(x_1)}{\Delta x} \]
When \(\Delta x\) is small enough
\[ \lim_{\Delta x\to0} \frac{f(x_1+ \Delta x) - f(x_1)}{\Delta x} = \frac{d}{dx} f(x) = f'(x) \]
Read \(\frac{d}{dx} f(x)\) as “the derivative of \(f\) of \(x\) with respect to \(x\)”
Read \(f'(x)\) as “\(f\) prime \(x\)” for a shorthand
\[ \lim_{\Delta x\to0} \frac{f(x_1+ \Delta x) - f(x_1)}{\Delta x} \]
\[ \lim_{h\to0} \frac{f(x_1+ h) - f(x_1)}{h} \]
Back to \(x^2\)
\[ \lim_{h \to0} \frac{f(x+ h)^2 - x^2}{h} \]
\[ \lim_{h\to0} \frac{f(x_1+ h) - f(x_1)}{h} \]
Back to \(x^2\)
\[ \lim_{h\to0} \frac{f(x+ h)^2 - x^2}{h} = \lim_{h\to0} \frac{x^2 + 2xh + 2h^2 - x^2}{h} \]
\[ \lim_{h\to0} \frac{f(x_1+ h) - f(x_1)}{h} \]
Back to \(x^2\)
\[ \lim_{h\to0} \frac{f(x+ h)^2 - x^2}{h} = \lim_{h\to0} \frac{x^2 + 2xh + 2h^2 - x^2}{h} = \lim_{h \to 0} \frac{2xh + 2h^2}{h} \]
\[ \lim_{h\to0} \frac{f(x_1+ h) - f(x_1)}{h} \]
Back to \(x^2\)
\[ \lim_{h\to0} \frac{f(x+ h)^2 - x^2}{h} = \lim_{h\to0} \frac{x^2 + 2xh + 2h^2 - x^2}{h} = \lim_{h \to 0} \frac{2xh + 2h^2}{h} \] \[ = \lim_{h \to 0} \frac{(2x+2h) h}{h} \]
\[ \lim_{h\to0} \frac{f(x_1+ h) - f(x_1)}{h} \]
Back to \(x^2\)
\[ \lim_{h\to0} \frac{f(x+ h)^2 - x^2}{h} = \lim_{h\to0} \frac{x^2 + 2xh + 2h^2 - x^2}{h} = \lim_{h \to 0} \frac{2xh + 2h^2}{h} \] \[ = \lim_{h \to 0} \frac{(2x+2h) h}{h} = \lim_{h \to 0} 2x+2h \]
\[ \lim_{h\to0} \frac{f(x_1+ h) - f(x_1)}{h} \]
Back to \(x^2\)
\[ \lim_{h\to0} \frac{f(x+ h)^2 - x^2}{h} = \lim_{h\to0} \frac{x^2 + 2xh + 2h^2 - x^2}{h} = \lim_{h \to 0} \frac{2xh + 2h^2}{h} \] \[ = \lim_{h \to 0} \frac{(2x+2h) h}{h} = \lim_{h \to 0} 2x+2h \] \[ =2x+0 \]
\[ \lim_{h\to0} \frac{f(x_1+ h) - f(x_1)}{h} \]
Back to \(x^2\)
\[ \lim_{h\to0} \frac{f(x+ h)^2 - x^2}{h} = \lim_{h\to0} \frac{x^2 + 2xh + 2h^2 - x^2}{h} = \lim_{h \to 0} \frac{2xh + 2h^2}{h} \] \[ = \lim_{h \to 0} \frac{(2x+2h) h}{h} = \lim_{h \to 0} 2x+2h \] \[ =2x+0 = 2x \]
Differentation: “Taking the derivative”
Constant rule: \((c)' = 0\).
\(y = 2\)
Coefficient rule: \((c \cdot f(x))' = c \cdot f'(x)\)
Sum/difference rule: \((f(x) \pm g(x))' = f'(x) \pm g'(x)\).
Example: \(f(x) = 3x + 7\)
\[ f'(x) = (3x + 7)' \]
\[ = (3x)' + (7)' \]
\[ = 3 + 0 \]
\[ = 3 \]
Power rule: \((x^n)'=nx^{(n-1)}\)
\[ f(x) = x^2 \]
\[ f'(x) = nx^{(n-1)} \]
\[ = 2x^{(2-1)} \]
\[ = 2x \]
Let’s try out \(\frac{d}{dx}4x^3\) and \(\frac{d}{dx}(x^2 + 2x)\) on the board.
Use the differentiation rules we have covered so far to calculate the derivatives of \(y\) with respect to \(x\) of as many as these as you can.
Exponent and logarithm rules:
\[ \begin{aligned} (c^x)' &= c^x \cdot ln(c), & \forall x>0 \\ \\ (log_a(x))' &= \frac{1}{x \cdot ln(a)}, & \forall x>0 \\ \end{aligned} \]
Exponent and logarithm rules:
Euler’s number (\(e\)) has interesting properties when it comes to derivatives.
\[ \begin{aligned} (e^x)' &= e^x \\ \\ (ln(x))' &= \frac{1}{x}, & \forall x>0 \end{aligned} \]
Compute the following:
Product rule: \((f(x)g(x))'=f'(x)g(x) + g'(x)f(x)\)
\[ [(3x+4)(x+2)]' \]
\[ f(x) = (3x+4), g(x) = (x+2); f'(x) = 3, g'(x) = 1 \]
\[ \color{blue}{f'(x)}\color{yellow}{g(x)} + \color{purple}{g'(x)}\color{green}{f(x)} \]
\[ = \color{blue}3 \color{yellow}{(x+2)} + \color{purple}{1} \color{green}{(3x+4)} \]
\[ = 3x + 6 + 3x+ 4 \]
\[ = 6x + 10 \]
Quotient rule: \(\displaystyle(\frac{f(x)}{g(x)})' = \frac{f'(x)g(x) + g'(x)f(x)}{[g(x)]^2}\)
\[ \frac{3x^2}{x+2} \]
Quotient rule: \(\displaystyle(\frac{f(x)}{g(x)})' = \frac{f'(x)g(x) + g'(x)f(x)}{[g(x)]^2}\)
\[ \frac{3x^2}{x+2} \Rightarrow f(x) = 3x^2, g(x) = x + 2; f'(x) = 6x, g'(x) = 1 \]
\[ \frac{\color{blue}{f'(x)}\color{yellow}{g(x)} + \color{purple}{g'(x)}\color{green}{f(x)}}{[\color{yellow}{g(x)}]^2} = \frac{\color{blue}{6x}\color{yellow}{(x+2)} - \color{purple}1 \color{green}{(3x^2)}}{\color{yellow}{(x+2)}^2} \]
\[ = \frac{6x^2 + 12x - 3x2}{(x+2)^2} = \frac{3x^2+12x}{(x+2)^2} \]
Chain rule: \((f(g(x))' = f'(g(x)) \cdot g'(x)\)
\[ f(g(x)) = (x+3)^3 \]
Chain rule: \([f(g(x))]' = f'(g(x)) \cdot g'(x)\)
\[ f(g(x)) = (x+3)^3 \Rightarrow f(g(x)) = g(x)^3, g(x) = (x+3) \]
\[ f'(x) = 3x^2 \Rightarrow f'(g(x)) = 3 (x+3)^2, g'(x) = 1 \]
Substitute
\[ [f(g(x))]' = 3(x+3)^2 \cdot 1 \]
If we were driving a car:
To get \(f''(x)\) just take the derivative of the output of \(f'(x)\)
What happens when we have more than one variable? \(f(x,z)\)
Partial derivative
\[ \frac{\partial}{\partial_x}f(x,z) = \frac{\partial_y}{\partial_x} = \partial_x f \]
Just treat other variables as constants and calculate the derivative of the target variable (usually \(x\))
\[ \begin{aligned} y = f(x,z) &= xz \\ \frac{\partial_y}{\partial_x} &= z \end{aligned} \]
Let’s try \(\displaystyle\frac{\partial (x^2y+xy^2-x)}{\partial x}\) and \(\displaystyle\frac{\partial (x^2y+xy^2-x)}{\partial y}\) together
optim() in R)torch package in R, various packages in Python)Find minimum or maximum values
Valleys are minima
Peaks are maxima
Both are extrema
Extrama can be local or global
Minimum: Derivative is negative before and positive after
Maximum: Derivative is positive before and negative after
Extrema are critical points
First order condition helps us find them
\[FOC: f'(x)=0\]
FOC finds critical points, but can’t tell if minimum or maximum
Second derivative test
Start by identifying \(f''(x)\)
Substitute in the stationary points \((x^*)\) identified from the FOC
\(f''(x^*) > 0\) we have a local minimum
\(f''(x^*) < 0\) we have a local maximum
\(f''(x^*) = 0\) we (may) have an inflection point - need to calculate higher-order derivatives
Obtain FOC and SOC for \(\displaystyle y=\frac{1}{2} x^3 + 3 x^2 - 2\)
Extreme value theorem:
If a real-valued function is continuous on a closed and bounded (finite) interval, the function must have a global minimum and a global maximum on that interval at least once
Either at the boundaries or at one of the local extrema
To find the minimum/maximum on some interval, compare the local min/max to the value of the function at the interval’s endpoints. So, if the interval is \((-\infty, +\infty)\), check the function’s limits as it approaches \(-\infty\) and \(+\infty\).